<html>
<head><meta charset="utf-8"><title>memory usage in borrowck · t-compiler/performance · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/index.html">t-compiler/performance</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html">memory usage in borrowck</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="223052916"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223052916" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223052916">(Jan 17 2021 at 17:06)</a>:</h4>
<p>hey <span class="user-mention" data-user-id="216206">@lcnr</span>, <code>Body</code> is really expensive to clone, right? I see it happening a lot in <a href="https://github.com/rust-lang/rust/blob/7d3818152d8ab5649d2e5cc6d7851ed7c03055fe/compiler/rustc_mir/src/borrow_check/mod.rs#L191">https://github.com/rust-lang/rust/blob/7d3818152d8ab5649d2e5cc6d7851ed7c03055fe/compiler/rustc_mir/src/borrow_check/mod.rs#L191</a>. Is there a way to avoid that?</p>
<p>context: <a href="https://github.com/rust-lang/rust/issues/81124">https://github.com/rust-lang/rust/issues/81124</a></p>



<a name="223053007"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223053007" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lcnr <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223053007">(Jan 17 2021 at 17:08)</a>:</h4>
<blockquote>
<p>I see it happening a lot </p>
</blockquote>
<p>once per query call? or am i missing something here</p>



<a name="223053012"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223053012" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223053012">(Jan 17 2021 at 17:08)</a>:</h4>
<p>once per function body I think</p>



<a name="223053018"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223053018" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lcnr <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223053018">(Jan 17 2021 at 17:09)</a>:</h4>
<p>yeah</p>



<a name="223053022"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223053022" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lcnr <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223053022">(Jan 17 2021 at 17:09)</a>:</h4>
<p>exactly</p>



<a name="223053026"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223053026" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223053026">(Jan 17 2021 at 17:09)</a>:</h4>
<p>which seems like a lot, that's double the memory, right?</p>



<a name="223053077"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223053077" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223053077">(Jan 17 2021 at 17:10)</a>:</h4>
<p>(I'm trying to hack things with <code>steal()</code> rn but I doubt they'll work)</p>



<a name="223053081"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223053081" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lcnr <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223053081">(Jan 17 2021 at 17:10)</a>:</h4>
<p>steal won't work</p>



<a name="223053084"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223053084" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lcnr <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223053084">(Jan 17 2021 at 17:10)</a>:</h4>
<p>at least i don't see how it should</p>



<a name="223053143"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223053143" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223053143">(Jan 17 2021 at 17:12)</a>:</h4>
<p>well, this might be on the wrong track anyway, -Z time-passes shows the memory being held all the way through linking and I think the clone should be dropped after borrow checking finishes</p>



<a name="223053148"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223053148" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223053148">(Jan 17 2021 at 17:12)</a>:</h4>
<p>is there a way to see what uses so much memory?</p>



<a name="223053157"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223053157" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lcnr <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223053157">(Jan 17 2021 at 17:12)</a>:</h4>
<p>we could probably avoid cloning the mir body here but it would be difficult i think</p>



<a name="223053168"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223053168" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lcnr <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223053168">(Jan 17 2021 at 17:13)</a>:</h4>
<p>maybe it's because we create a lot of new types there?</p>



<a name="223053182"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223053182" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lcnr <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223053182">(Jan 17 2021 at 17:14)</a>:</h4>
<p>i don't know too much about profiling the compiler</p>



<a name="223053219"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223053219" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lcnr <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223053219">(Jan 17 2021 at 17:14)</a>:</h4>
<p>so i can't help you much here</p>



<a name="223053224"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223053224" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223053224">(Jan 17 2021 at 17:14)</a>:</h4>
<p>I'll try heaptrack maybe</p>



<a name="223053320"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223053320" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223053320">(Jan 17 2021 at 17:17)</a>:</h4>
<p><span class="user-mention silent" data-user-id="216206">lcnr</span> <a href="#narrow/stream/247081-t-compiler.2Fperformance/topic/memory.20usage.20in.20borrowck/near/223053081">said</a>:</p>
<blockquote>
<p>steal won't work</p>
</blockquote>
<p>indeed you are correct</p>
<div class="codehilite"><pre><span></span><code>thread &#39;rustc&#39; panicked at &#39;attempt to read from stolen value&#39;, /home/joshua/rustc/compiler/rustc_data_structures/src/steal.rs:43:15
</code></pre></div>



<a name="223053328"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223053328" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223053328">(Jan 17 2021 at 17:17)</a>:</h4>
<p>I'll go add track_caller to that</p>



<a name="223053419"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223053419" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223053419">(Jan 17 2021 at 17:19)</a>:</h4>
<p>I think I've seen ConstProp to also clone the MIR body, but only part of it. Maybe borrowck could go that way if it doesn't need <em>everything</em>?</p>



<a name="223053692"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223053692" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223053692">(Jan 17 2021 at 17:26)</a>:</h4>
<p>heaptrack is hanging so I don't have better numbers unfortunately :(</p>



<a name="223053773"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223053773" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223053773">(Jan 17 2021 at 17:28)</a>:</h4>
<p>opened <a href="https://github.com/rust-lang/rust/pull/81125/">https://github.com/rust-lang/rust/pull/81125/</a></p>



<a name="223053927"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223053927" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223053927">(Jan 17 2021 at 17:32)</a>:</h4>
<p>ugh this was working last release</p>



<a name="223053930"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223053930" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223053930">(Jan 17 2021 at 17:32)</a>:</h4>
<p>I don't know what changed</p>



<a name="223054594"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223054594" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223054594">(Jan 17 2021 at 17:49)</a>:</h4>
<p>I tried installing the latest version of heaptrack without success</p>



<a name="223054600"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223054600" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223054600">(Jan 17 2021 at 17:49)</a>:</h4>
<p>why is all software buggy <span aria-label="sob" class="emoji emoji-1f62d" role="img" title="sob">:sob:</span></p>



<a name="223055067"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223055067" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223055067">(Jan 17 2021 at 18:00)</a>:</h4>
<p>opened <a href="https://bugs.kde.org/show_bug.cgi?id=431746">https://bugs.kde.org/show_bug.cgi?id=431746</a></p>



<a name="223056063"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223056063" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223056063">(Jan 17 2021 at 18:24)</a>:</h4>
<p>I wonder if we could use jemalloc's profiler?</p>



<a name="223056673"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223056673" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jonas Schievink  [he/him] <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223056673">(Jan 17 2021 at 18:41)</a>:</h4>
<p><span class="user-mention silent" data-user-id="232545">Joshua Nelson</span> <a href="#narrow/stream/247081-t-compiler.2Fperformance/topic/memory.20usage.20in.20borrowck/near/223053026">said</a>:</p>
<blockquote>
<p>which seems like a lot, that's double the memory, right?</p>
</blockquote>
<p>The second copy shouldn't be retained, at least. It's local to the MIR borrow-checker.</p>



<a name="223056726"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223056726" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jonas Schievink  [he/him] <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223056726">(Jan 17 2021 at 18:42)</a>:</h4>
<p>It does show up in profiles, though, which is why I've briefly looked into that</p>



<a name="223057953"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223057953" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223057953">(Jan 17 2021 at 19:09)</a>:</h4>
<p><span class="user-mention silent" data-user-id="334260">Dániel Buga</span> <a href="#narrow/stream/247081-t-compiler.2Fperformance/topic/memory.20usage.20in.20borrowck/near/223056063">said</a>:</p>
<blockquote>
<p>I wonder if we could use jemalloc's profiler?</p>
</blockquote>
<p>do you know how to do that?</p>



<a name="223058025"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223058025" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223058025">(Jan 17 2021 at 19:10)</a>:</h4>
<p>found why I thought heaptrack used to work: <a href="https://bugs.kde.org/show_bug.cgi?id=431746#c3">https://bugs.kde.org/show_bug.cgi?id=431746#c3</a></p>



<a name="223058098"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223058098" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223058098">(Jan 17 2021 at 19:12)</a>:</h4>
<p>The compiler needs to be recompiled agains jemalloct, with jemalloc-sys's profile feature enabled, and then it <em>should</em> be as simple as writing an env var... but I never tried, it's just what I gathered from the webz</p>



<a name="223058109"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223058109" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223058109">(Jan 17 2021 at 19:13)</a>:</h4>
<p>oh nice, <code>jemalloc</code> is an option in config.toml</p>



<a name="223058118"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223058118" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223058118">(Jan 17 2021 at 19:13)</a>:</h4>
<p>yup, I think it's there anyway, because the compiler reacts to the env vars it just doesn't recognize them as valid</p>



<a name="223058121"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223058121" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223058121">(Jan 17 2021 at 19:13)</a>:</h4>
<p>(these: <a href="https://github.com/jemalloc/jemalloc/wiki/Use-Case%3A-Heap-Profiling">https://github.com/jemalloc/jemalloc/wiki/Use-Case%3A-Heap-Profiling</a>)</p>



<a name="223059419"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223059419" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223059419">(Jan 17 2021 at 19:42)</a>:</h4>
<p>I tried the wiki but it broke:</p>
<div class="codehilite"><pre><span></span><code>&gt; MALLOC_CONF=prof_leak:true,lg_prof_sample:0,prof_final:true /home/joshua/.local/lib/rustup/toolchains/stage1/bin/rustc --version
&lt;jemalloc&gt;: Invalid conf pair: prof_leak:true
&lt;jemalloc&gt;: Invalid conf pair: lg_prof_sample:0
&lt;jemalloc&gt;: Invalid conf pair: prof_final:true
</code></pre></div>



<a name="223059491"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223059491" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223059491">(Jan 17 2021 at 19:44)</a>:</h4>
<p>Did you enable the profile feature for jemalloc-sys? it's not on by default</p>



<a name="223059500"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223059500" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223059500">(Jan 17 2021 at 19:44)</a>:</h4>
<p>I think it's in rustc's Cargo.toml</p>



<a name="223059509"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223059509" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223059509">(Jan 17 2021 at 19:44)</a>:</h4>
<p>oh hmm I did not</p>



<a name="223059517"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223059517" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223059517">(Jan 17 2021 at 19:45)</a>:</h4>
<p>I do not see a 'profile' feature</p>



<a name="223059523"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223059523" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223059523">(Jan 17 2021 at 19:45)</a>:</h4>
<p>do you mean jemalloc in general? I did enable that</p>



<a name="223059545"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223059545" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223059545">(Jan 17 2021 at 19:45)</a>:</h4>
<div class="codehilite" data-code-language="Diff"><pre><span></span><code><span class="gd">- jemalloc = ['jemalloc-sys']</span>
<span class="gi">+ jemalloc = ['jemalloc-sys/profiling']</span>
</code></pre></div>



<a name="223059587"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223059587" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223059587">(Jan 17 2021 at 19:46)</a>:</h4>
<p>oh I see it's not in cargo.toml at all</p>



<a name="223059590"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223059590" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223059590">(Jan 17 2021 at 19:46)</a>:</h4>
<p>that's silly</p>



<a name="223059598"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223059598" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223059598">(Jan 17 2021 at 19:46)</a>:</h4>
<p><del>https://github.com/gnzlbg/jemallocator/blob/master/Cargo.toml#L49</del> <a href="https://github.com/gnzlbg/jemallocator/blob/master/jemalloc-sys/Cargo.toml#L42">https://github.com/gnzlbg/jemallocator/blob/master/jemalloc-sys/Cargo.toml#L42</a> sorry it's profiling</p>



<a name="223059677"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223059677" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223059677">(Jan 17 2021 at 19:48)</a>:</h4>
<p>ayyy that did it, thanks :)</p>



<a name="223059989"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223059989" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223059989">(Jan 17 2021 at 19:55)</a>:</h4>
<p>oh lol I did leak checking, not heap profiling</p>



<a name="223059994"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223059994" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223059994">(Jan 17 2021 at 19:56)</a>:</h4>
<p>found leaks in LLVM</p>



<a name="223060124"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223060124" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223060124">(Jan 17 2021 at 19:59)</a>:</h4>
<p>... it looks like <code> MALLOC_CONF=prof:true,prof_prefix:jeprof.out,prof_final:true </code> didn't actually generate a file?</p>



<a name="223060203"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223060203" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223060203">(Jan 17 2021 at 20:01)</a>:</h4>
<div class="codehilite"><pre><span></span><code>&gt;  MALLOC_CONF=lg_prof_interval:1024,prof_prefix:jeprof.out,prof_final:true rustc
&lt;jemalloc&gt;: Out-of-range conf value: lg_prof_interval:1024
</code></pre></div>



<a name="223060217"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223060217" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223060217">(Jan 17 2021 at 20:01)</a>:</h4>
<p>damn it people I just want to generate a heap profile</p>



<a name="223060270"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223060270" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223060270">(Jan 17 2021 at 20:02)</a>:</h4>
<p>gotta love feature flags... I'm not sure about the output file</p>



<a name="223060293"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223060293" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223060293">(Jan 17 2021 at 20:03)</a>:</h4>
<p>well <code> MALLOC_CONF=prof_leak:true,lg_prof_sample:0,prof_final:true </code> did actually work so I'm confused why profiling doesn't</p>



<a name="223060456"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223060456" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223060456">(Jan 17 2021 at 20:06)</a>:</h4>
<p>oh hold on, the interval is log base 2, it was trying to record the heap every 2^1024 bytes lol</p>



<a name="223060555"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223060555" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223060555">(Jan 17 2021 at 20:08)</a>:</h4>
<p>nope, still not generating a file</p>



<a name="223060654"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223060654" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223060654">(Jan 17 2021 at 20:11)</a>:</h4>
<p>well anyway, I've spent enough time on this</p>



<a name="223061222"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223061222" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223061222">(Jan 17 2021 at 20:25)</a>:</h4>
<blockquote>
<p>It looks like the high memory usage is being driven by the MIR for the TRANSITIONS static: <a href="https://github.com/wez/wezterm/blob/fa4bbbd077837d4eafe26805079f95a51054c4d7/vtparse/src/transitions.rs#L300-L315">https://github.com/wez/wezterm/blob/fa4bbbd077837d4eafe26805079f95a51054c4d7/vtparse/src/transitions.rs#L300-L315</a></p>
</blockquote>
<p><span class="user-mention" data-user-id="125294">@Aaron Hill</span> how did you find that out?</p>



<a name="223061228"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223061228" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223061228">(Jan 17 2021 at 20:25)</a>:</h4>
<p>I first ran the compiler under massif</p>



<a name="223061233"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223061233" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223061233">(Jan 17 2021 at 20:25)</a>:</h4>
<p>massif, ok</p>



<a name="223061235"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223061235" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223061235">(Jan 17 2021 at 20:25)</a>:</h4>
<p>everything else I tried has been buggy</p>



<a name="223061238"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223061238" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223061238">(Jan 17 2021 at 20:25)</a>:</h4>
<p>confusingly, that changed the <code>rss</code> values printed out by the compiler</p>



<a name="223061252"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223061252" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223061252">(Jan 17 2021 at 20:25)</a>:</h4>
<p>yeah, heaptrack hung with zero cpu usage at the end of compilation</p>



<a name="223061298"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223061298" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223061298">(Jan 17 2021 at 20:26)</a>:</h4>
<p>yup, I reported that bug: <a href="https://bugs.kde.org/show_bug.cgi?id=431746">https://bugs.kde.org/show_bug.cgi?id=431746</a></p>



<a name="223061299"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223061299" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223061299">(Jan 17 2021 at 20:26)</a>:</h4>
<p>anyway, that showed high memory usage in the dataflow <code>Engine</code>, due to allocating a large IndexVec</p>



<a name="223061314"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223061314" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223061314">(Jan 17 2021 at 20:26)</a>:</h4>
<p>I added in some printlns, and discovered the MIR body with a huge number of basic blocks</p>



<a name="223061425"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223061425" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223061425">(Jan 17 2021 at 20:29)</a>:</h4>
<p>Here's the output of <code>cargo expand</code>: <a href="https://gist.github.com/Aaron1011/73c3f35c6aa1b07bf5a5d2240bc71b19">https://gist.github.com/Aaron1011/73c3f35c6aa1b07bf5a5d2240bc71b19</a></p>



<a name="223061434"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223061434" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223061434">(Jan 17 2021 at 20:29)</a>:</h4>
<p>this is just an enormous initializer</p>



<a name="223061439"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223061439" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223061439">(Jan 17 2021 at 20:29)</a>:</h4>
<p>wow, that is indeed a ludicrous number of basic blocks</p>



<a name="223061454"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223061454" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223061454">(Jan 17 2021 at 20:29)</a>:</h4>
<p>I don't think we can possibly do better than one basic block per function call</p>



<a name="223061455"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223061455" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223061455">(Jan 17 2021 at 20:29)</a>:</h4>
<p>is there a way to reduce the amount of memory each block uses?</p>



<a name="223061456"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223061456" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223061456">(Jan 17 2021 at 20:29)</a>:</h4>
<p>since we need to handle unwinding</p>



<a name="223061503"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223061503" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223061503">(Jan 17 2021 at 20:30)</a>:</h4>
<p>so I think that's going to limit how much we can improve memory usage</p>



<a name="223061609"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223061609" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223061609">(Jan 17 2021 at 20:32)</a>:</h4>
<p>this might be a good benchmark to add, if rustc-timer has the time budget</p>



<a name="223061614"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223061614" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223061614">(Jan 17 2021 at 20:32)</a>:</h4>
<p>if we just add the initializer I think it wouldn't take too much time</p>



<a name="223061820"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223061820" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223061820">(Jan 17 2021 at 20:37)</a>:</h4>
<p>I don't know where in the optimization pipeline we do const-prop</p>



<a name="223061849"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223061849" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223061849">(Jan 17 2021 at 20:37)</a>:</h4>
<p>but there could be some benefit to trying to identify and collapse things like 'litint + lintint * lintint' very early on</p>



<a name="223061896"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223061896" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223061896">(Jan 17 2021 at 20:38)</a>:</h4>
<p>as I know that's a common pattern for macro_rules macros to build up an index value</p>



<a name="223062653"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223062653" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223062653">(Jan 17 2021 at 20:56)</a>:</h4>
<p>Well this is no solution but probably didn't help either: <a href="https://github.com/rust-lang/rust/pull/81132">https://github.com/rust-lang/rust/pull/81132</a> ~50k allocations one by one? <span aria-label="smile" class="emoji emoji-1f642" role="img" title="smile">:smile:</span> (to clarify, it's a lot less allocations because of the growth algo)</p>



<a name="223062670"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223062670" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223062670">(Jan 17 2021 at 20:57)</a>:</h4>
<p><span class="user-mention" data-user-id="334260">@Dániel Buga</span> the time to execute is actually not bad, it's the peak RSS that's off the charts</p>



<a name="223062676"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223062676" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223062676">(Jan 17 2021 at 20:57)</a>:</h4>
<p>so pre-allocating wouldn't help much</p>



<a name="223062685"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223062685" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223062685">(Jan 17 2021 at 20:57)</a>:</h4>
<p>Depends on what happens on growth</p>



<a name="223062726"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223062726" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223062726">(Jan 17 2021 at 20:58)</a>:</h4>
<p>Anyway, may not even get to this point, but still, since we know the upper bound of the hash map we need, why not reserve it?</p>



<a name="223062827"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223062827" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223062827">(Jan 17 2021 at 21:01)</a>:</h4>
<p>right, it's a good change, I just don't think it will help in this case</p>



<a name="223062842"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223062842" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223062842">(Jan 17 2021 at 21:01)</a>:</h4>
<p>Just like I said <span aria-label="wink" class="emoji emoji-1f609" role="img" title="wink">:wink:</span></p>



<a name="223064592"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223064592" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223064592">(Jan 17 2021 at 21:36)</a>:</h4>
<p>Well, a BasicBlockData is 144 bytes, out of which Terminator is 112 bytes - so I guess the Terminator can be annoying here. But for 30k basic blocks in that expansion, this is only 4MB of memory or so. We could cut that down by boxing the larger variants, but there might be a runtime price for that</p>



<a name="223064844"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223064844" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223064844">(Jan 17 2021 at 21:40)</a>:</h4>
<p>But in this case it's a series of Asserts and Calls, both of which are rather fat, so just by boxing, we wouldn't even win much here, if anything.</p>



<a name="223069682"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223069682" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223069682">(Jan 17 2021 at 23:35)</a>:</h4>
<p>opened <a href="https://github.com/rust-lang/rust/pull/81140">https://github.com/rust-lang/rust/pull/81140</a> just in case it helps (and if it works, lol)</p>



<a name="223069741"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223069741" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223069741">(Jan 17 2021 at 23:37)</a>:</h4>
<p><span class="user-mention" data-user-id="334260">@Dániel Buga</span> can you run that on vtparse and see if it helps memory usage at all?</p>



<a name="223069871"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223069871" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223069871">(Jan 17 2021 at 23:40)</a>:</h4>
<p>sorry its 0:40, I'm done for yesterday <span aria-label="smile" class="emoji emoji-1f642" role="img" title="smile">:smile:</span></p>



<a name="223069879"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223069879" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223069879">(Jan 17 2021 at 23:41)</a>:</h4>
<p>my guess is no, though</p>



<a name="223070080"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223070080" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223070080">(Jan 17 2021 at 23:47)</a>:</h4>
<p>fair enough lol</p>



<a name="223073547"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223073547" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223073547">(Jan 18 2021 at 01:23)</a>:</h4>
<p><span class="user-mention silent" data-user-id="232545">Joshua Nelson</span> <a href="#narrow/stream/247081-t-compiler.2Fperformance/topic/memory.20usage.20in.20borrowck/near/223053328">said</a>:</p>
<blockquote>
<p>I'll go add track_caller to that</p>
</blockquote>
<p>Do we know how often is it used? (And do we have an intuition for what the cost of <code>#[track_caller]</code> is for that matter?)</p>



<a name="223073593"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223073593" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223073593">(Jan 18 2021 at 01:24)</a>:</h4>
<p>the way <code>#[track_caller]</code> codegens smells like it is likely to cause runtime regressions, but <span aria-label="shrug" class="emoji emoji-1f937" role="img" title="shrug">:shrug:</span></p>



<a name="223073597"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223073597" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223073597">(Jan 18 2021 at 01:24)</a>:</h4>
<p>why does track_caller cause runtime regressions?</p>



<a name="223073599"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223073599" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223073599">(Jan 18 2021 at 01:24)</a>:</h4>
<p>steal is not used much at all:</p>
<div class="codehilite"><pre><span></span><code>compiler/rustc_typeck/src/collect/type_of.rs
651:    match tcx.sess.diagnostic().steal_diagnostic(span, StashKey::ItemNoType) {

compiler/rustc_mir/src/transform/mod.rs
262:    let mut body = tcx.mir_built(def).steal();
297:    let mut body = tcx.mir_const(def).steal();
423:    let mut body = body.steal();
568:        tcx.mir_drops_elaborated_and_const_checked(ty::WithOptConstParam::unknown(did)).steal();
592:    let mut promoted = promoted.steal();

compiler/rustc_interface/src/queries.rs
218:            let resolver = peeked.1.steal();
265:                resolver_outputs.steal(),
</code></pre></div>



<a name="223073608"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223073608" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223073608">(Jan 18 2021 at 01:25)</a>:</h4>
<p>I did not say it does, I said it smells like it would. That's because it unconditionally passes of location data to a function (+register/stack pressure).</p>



<a name="223073618"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223073618" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223073618">(Jan 18 2021 at 01:25)</a>:</h4>
<p>borrow is used a lot more:</p>
<div class="codehilite"><pre><span></span><code>&gt; rg &#39;\.borrow\(&#39; compiler/ | wc -l
227
</code></pre></div>



<a name="223073656"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223073656" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223073656">(Jan 18 2021 at 01:26)</a>:</h4>
<p>do you want me to run perf on that PR before merging it?</p>



<a name="223073662"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223073662" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223073662">(Jan 18 2021 at 01:26)</a>:</h4>
<p>nah, I think the uses of <code>steal</code> are rare and reasonably cold enough that it wouldn't be necessary.</p>



<a name="223073666"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223073666" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223073666">(Jan 18 2021 at 01:26)</a>:</h4>
<p>oh well I added <code>track_caller</code> to <code>borrow</code> too <span aria-label="sweat smile" class="emoji emoji-1f605" role="img" title="sweat smile">:sweat_smile:</span></p>



<a name="223073678"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223073678" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223073678">(Jan 18 2021 at 01:27)</a>:</h4>
<p>Might be worth it at some point down the line to remove all #[track_caller] and perf to quantify.</p>



<a name="223073680"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223073680" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223073680">(Jan 18 2021 at 01:27)</a>:</h4>
<p>if it's a big regression it will come up in the weekly triage report in any case</p>



<a name="223073681"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223073681" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223073681">(Jan 18 2021 at 01:27)</a>:</h4>
<p>so I can revert it then if necessary</p>



<a name="223084520"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223084520" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tm <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223084520">(Jan 18 2021 at 06:23)</a>:</h4>
<p>This crate looks like something that would encounter similar issues like those described in <a href="https://github.com/rust-lang/rust/issues/54208#issuecomment-421188735">https://github.com/rust-lang/rust/issues/54208#issuecomment-421188735</a></p>



<a name="223352467"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223352467" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223352467">(Jan 20 2021 at 10:06)</a>:</h4>
<p>(deleted)</p>



<a name="223407422"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223407422" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223407422">(Jan 20 2021 at 17:38)</a>:</h4>
<p>Fun fact: I tried to skip the borrow checking dataflow pass because the code in question doesn't have a single borrow and that pass allocates a 33k x size_of(empty BitSet) array. Turns out, by doing that, I managed somehow to increase max rss by 200MB. <span aria-label="sweat smile" class="emoji emoji-1f605" role="img" title="sweat smile">:sweat_smile:</span></p>



<a name="223408963"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223408963" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223408963">(Jan 20 2021 at 17:49)</a>:</h4>
<p>There are 124559 move paths in this particular piece of code. That, with the 33k basic blocks means that each dataflow pass (MaybeUninitializedPlaces, MaybeInitializedPlaces, EverInitializedPlaces) allocates like 500MB (I think) of memory if I'm not completely wrong. I guess, maybe in this particular case a sparse representation of the dataflow state would help?</p>



<a name="223409717"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223409717" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jonas Schievink  [he/him] <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223409717">(Jan 20 2021 at 17:54)</a>:</h4>
<p>I think we have a sparse bitset introduced for borrowck already?</p>



<a name="223410226"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223410226" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223410226">(Jan 20 2021 at 17:58)</a>:</h4>
<p>we have a number of sparse bitset data structures, yes</p>



<a name="223410297"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223410297" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tm <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223410297">(Jan 20 2021 at 17:58)</a>:</h4>
<p>There is also a hybrid one that combines the two.</p>



<a name="223410325"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223410325" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223410325">(Jan 20 2021 at 17:59)</a>:</h4>
<p>Yes, GenKillSet is implemented in terms of the HybridBitSet, and it is used when the DFG is cyclic as well.</p>



<a name="223411969"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223411969" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tm <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223411969">(Jan 20 2021 at 18:10)</a>:</h4>
<p>At some point in the computation those sets become quite dense or even start as such, so that's probably not that easy to improve.</p>



<a name="223414812"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223414812" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jonas Schievink  [he/him] <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223414812">(Jan 20 2021 at 18:31)</a>:</h4>
<p>Why are there so many move paths in the first place?</p>



<a name="223414833"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223414833" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223414833">(Jan 20 2021 at 18:31)</a>:</h4>
<p>hopefully not for every single basic block of a humongous static initializer <span aria-label="smile" class="emoji emoji-1f642" role="img" title="smile">:smile:</span></p>



<a name="223414913"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223414913" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223414913">(Jan 20 2021 at 18:31)</a>:</h4>
<p>the mir looks like this: <a href="https://rust.godbolt.org/z/hPdGrK">https://rust.godbolt.org/z/hPdGrK</a></p>



<a name="223415035"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223415035" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223415035">(Jan 20 2021 at 18:32)</a>:</h4>
<p>(the example is reduced to 1/14 of the original)</p>



<a name="223545812"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223545812" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223545812">(Jan 21 2021 at 17:53)</a>:</h4>
<p><span class="user-mention silent" data-user-id="232545">Joshua Nelson</span> <a href="#narrow/stream/247081-t-compiler.2Fperformance/topic/memory.20usage.20in.20borrowck/near/223053692">said</a>:</p>
<blockquote>
<p>heaptrack is hanging so I don't have better numbers unfortunately :(</p>
</blockquote>
<p>oh hey, they got back on this <a href="https://bugs.kde.org/show_bug.cgi?id=431746#c4">https://bugs.kde.org/show_bug.cgi?id=431746#c4</a></p>



<a name="223552413"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223552413" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223552413">(Jan 21 2021 at 18:41)</a>:</h4>
<p>(deleted)</p>



<a name="223553431"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/memory%20usage%20in%20borrowck/near/223553431" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/memory.20usage.20in.20borrowck.html#223553431">(Jan 21 2021 at 18:49)</a>:</h4>
<p>(deleted)</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>